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ABSTRACT 



This report provides background, implementation, and diverse 
outcome data from the first 3 years of an ongoing effort to implement a 
highly specified school reform design in a high-poverty, urban elementary 
school, Woodson Elementary School, Baltimore (Maryland). The design that is 
being implemented is the Calvert School model. The Calvert School is a 
private elementary school with a long history of providing a high-quality 
education to many of Baltimore's most affluent families. Many children 
worldwide are home- schooled using Calvert's highly structured home 
instruction program. High expectations, time-on-task, rapid pace of 
instruction, frequent evaluations, immediate feedback, and student accuracy 
are viewed as fundamental in the Calvert model . In a method that predated 
portfolio instruction by over half a century, Calvert produces a "book" of 
each student's school folder for each month of the school year. The Calvert 
model had already been implemented at one Baltimore school with a great deal 
of success when the program was implemented at Woodson. The Woodson 
Elementary School is located in a predominantly African American community in 
Baltimore's central city. Over 90% of students are eligible for free or 
reduced-price lunch. Third-year implementation was largely, although not 
totally, successful. Outcome data indicate that the Calvert program has had a 
positive academic impact on students attending Woodson Elementary School . The 
quality of work in students' folders has been high, and student attendance 
has averaged 96.4%. Referral rates to special education remain low, and 
end-of -third-grade Comprehensive Testing Program scores have been 
dramatically above those of previous Woodson students . Scores on the Maryland 
School Performance Assessment Program have also improved. (Contains 13 tables 
and 22 references.) (SLD) 
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The Center 



Every child has the capacity to succeed in school and in life. Yet far too many 
children, especially those from poor and minority families, are placed at risk by school 
practices that are based on a sorting paradigm in which some students receive high-expecta- 
tions instruction while the rest are relegated to lower quality education and lower quality 
futures. The sorting perspective must be replaced by a “talent development” model that 
asserts that all children are capable of succeeding in a rich and demanding curriculum with 
appropriate assistance and support. 

The mission of the Center for Research on the Education of Students Placed At Risk 
(CRESPAR) is to conduct the research, development, evaluation, and dissemination needed 
to transform schooling for students placed at risk. The work of the Center is guided by three 
central themes — ensuring the success of all students at key development points, building 
on students’ personal and cultural assets, and scaling up effective programs — and conducted 
through seven research and development programs and a program of institutional activities. 

CRESPAR is organized as a partnership of Johns Hopkins University and Howard 
University, in collaboration with researchers at the University of California at Santa Barbara, 
University of California at Los Angeles, University of Chicago, Manpower Demonstration 
Research Corporation, University of Memphis, Haskell Indian Nations University, and 
University of Houston-Clear Lake. 

CRESPAR is supported by the National Institute on the Education of At-Risk 
Students (At-Risk Institute), one of five institutes created by the Educational Research, 
Development, Dissemination and Improvement Act of 1994 and located within the Office 
of Educational Research and Improvement (OERI) at the U.S. Department of Education. The 
At-Risk Institute supports a range of research and development activities designed to 
improve the education of students at risk of educational failure because of limited English 
proficiency, poverty, race, geographic location, or economic disadvantage. 
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Abstract 



This report provides background, implementation, and diverse outcome data from the 
first three years of an ongoing effort to implement a highly specified school reform design 
in a high-poverty, urban elementary school. The design that is being implemented is the 
Calvert School model, and that model is described in this report. Third-year implementation 
was largely, though not totally, successful. Outcome data indicate that the Calvert program 
has had a positive academic impact on students attending Woodson Elementary School. 
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Introduction 



The two largest U S. multi-year, outcome-reporting studies of deliberate school 
change efforts of the last 20 years have both concluded that the local implementa- 
tion/adaptation of externally developed programs have tended to have greater impacts on 
student achievements than have locally developed programs (Crandall et al., 1982; 
Stringfield et al., 1997). Further, both the Follow-Through evaluations (Stallings & 
Kaskowitz, 1974) and the evaluations of Success for All (Slavin et al., 1996) have tended to 
indicate that, in the words of Rose (1995), “Combine high standards with structure and you 
get ... reform that works.” If these findings and assertions are correct, the careful introduction 
of a highly specified, high-content design into an inner-city elementary school should 
produce substantial academic gains. 

This technical report provides data from a three-year case study of such an 
experiment. The report describes the Calvert School program, the Woodson Elementary 
School in which the program was implemented, and the process of implementation. It then 
provides detailed outcome data that include attendance, scores on norm-referenced tests, and 
scores on the Maryland School Performance Assessment Program (MSPAP). 



Calvert School 

Calvert is a private elementary school with a long history of providing a high-quality 
education to several generations of children from many of Baltimore’s most affluent 
families. Begun in 1897, the school now educates 390 children from prekindergarten through 
grade six in its day school. (Another 16,000 children worldwide are home-schooled using 
Calvert’s highly structured home instruction program.) Tuition at Calvert School averages 
over $9,000 per year. 

Calvert carefully selects students for admittance to its day school from children who 
perform well on an aptitude test and in an interview. During the 1996-1997 school year, 90% 
of the students attending Calvert were White, 6% were African-American, and 4% were 
Asian or Hispanic. Fifteen percent of Calvert’s students receive partial to nearly full financial 
assistance. 

A coherent philosophy of education pervades Calvert. High expectations, time-on- 
task, rapid pace of instruction, frequent evaluations, immediate feedback, and student 
accuracy are all viewed as fundamental. The students are required to leam attention to detail, 
including correct spelling and punctuation. Parents receive monthly report cards and 



representative samples of the student’s academic work for the month. In a practice that 
predated “portfolio assessment” by over half a century, Calvert produces a “book” of each 
student’s nine monthly “folders” of work and presents the book to the student at the end of 
each year. 

Many observers might characterize the Calvert curriculum as “traditional,” however, 
the label would be superficial. Early grades’ reading instruction focuses of getting meaning 
out of reading. The school formally teaches sight words and phonemic skills. Students are 
taught to read for specific purposes, and there is also time during each school day for reading 
for enjoyment. Students make frequent use of mathematics manipulatives and do timed fact 
drills on basic mathematics facts. Calvert has always taught much of the history, literature, 
geography, science, art, art history, and music that comprise E.D. Hirsch’s (1987, 1996) Core 
Knowledge curriculum. 



Woodson Elementary School 

Carter G. Woodson Elementary School is located in a predominantly African- 
American community near, but separated by water from Baltimore’s central business district. 
Over 90% of Woodson students are eligible for free or reduced-price lunch. This percentage 
is nearly triple the 1996 Maryland state average (31.1%) and well above the Baltimore City 
average of 70.4%. In the early 1990s, the school had a stable record of low student 
achievement. 



A Brief History of the Calvert Program 
in the Baltimore City Schools 

The idea to put the curriculum of the Calvert School, a private Baltimore elementary 
school, into urban public schools began nearly 20 years ago. Muriel Berkeley (now director 
of the Baltimore Curriculum Project) met with a “Cities in Schools” representative who had 
moved to Washington, D.C. to work in the Carter administration. During their discussions 
the man volunteered that his children were being home-schooled with the Calvert School 
Home Instruction Course. The two talked about the merits of the Calvert education, and the 
man suggested to Dr. Berkeley that the use of the Calvert curriculum could increase the 
academic achievement of students in urban public schools. Dr. Berkeley contacted William 
Kirk, then the Head Master of the Calvert School, to see if Calvert would be amenable to 
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such an experiment. Mr. Kirk communicated the school’s willingness to discuss participation 
in the proposed reform. 

Representatives from the Washington, D.C. school district and the representative of 
Cities in Schools met with Calvert officials. It was eventually decided that the Calvert 
curriculum was not appropriate for the D.C. schools. A regulation existed at the time in that 
district that prohibited students from using materials that were part of the curriculum of a 
lower grade. This was incompatible with the Calvert philosophy that children should receive 
instruction at their achievement level, not their grade level. 

At the same time, Dr. Berkeley suggested the use of the Calvert curriculum to three 
Baltimore principals, Kathy Pope Smith of Federal Hill Elementary, Gertrude Williams of 
Barclay Elementary and Middle School, and Evelyn Beasley of Roland Park Elementary and 
Middle School. Dr. Berkeley and the principals visited Calvert and met with Mr. Kirk. 
However, concrete plans to place the Calvert Program in a Baltimore City school would wait 
for nearly three years. 

In 1983, Mrs. Smith of Federal Hill Elementary renewed her interest in Calvert. She 
met twice with Merrill T. Hall, III, the new Calvert Head Master. The discussions at these 
meetings led to Mrs. Smith submitting a proposal to the Baltimore Fund for Educational 
Excellence to finance the implementation of the Calvert Program at her school. The Fund 
approved her proposal. However, Mrs. Smith was met with firm resistance from the then- 
superintendent of Baltimore City Public Schools. The superintendent forbade the use of the 
Calvert curriculum in any city public school, based on her belief that the city’s curriculum 
was superior. The school district put an end to Mrs. Smith’s plan to bring Calvert to Federal 
Hill. 



Five years later, in 1988, senior-level representatives of The Abell Foundation in 
Baltimore visited Calvert to learn about the school’s educational philosophy and program. 
After one visit to Calvert, the foundation’s president asked Mr. Hall if the Calvert School 
would be willing to participate in an experiment in which the Calvert curriculum would be 
placed in a Baltimore public school. Mr. Hall found the concept provocative and proposed 
the idea to the Calvert Board of Directors. The board approved the concept. 

Abell contacted Mrs. Williams, principal of Barclay (Mrs. Smith had died in the 
intervening years), to gauge her interest in participating in the experiment. She readily 
agreed. Extended discussions with the new Baltimore City Schools Superintendent and the 
School Board followed, and the process eventually resulted in Board approval of the project. 
Nearly 10 years following the genesis of the idea to place the private school curriculum of 
Calvert School into an urban public school, the elements necessary to effect the reform were 



in place. The Calvert School provided the program, The Abell Foundation provided the 
funding, Barclay Elementary and Middle School provided the school for experimentation, 
and the Baltimore City Public Schools consented to the project. In September 1989 the first 
cohort of Barclay students began using the Calvert curriculum (Edmunds, 1993; Rose, 1995). 

After evaluators reported four consecutive years of positive results in student 
achievement at Barclay (see Stringfield, 1995), Abell and Calvert decided to expand to one 
additional Baltimore City public school. The goal was to verify that the success experienced 
at Barclay could be replicated. 

Concurrently, Susan Spath, principal of Carter G. Woodson Elementary, a Baltimore 
public school, began to pursue the possibility of using the Calvert curriculum in her school 
after reading of the success of the reform at Barclay. Dr. Spath’s and Calvert’s parallel 
interests blended and Woodson was selected as the second school to implement the Calvert 
curriculum. In the summer of 1994 the first group of Woodson teachers and assistants 
received training in the Calvert curriculum and processes, and in September 1994 the first 
cohort of Woodson kindergarten and first-grade students began receiving instruction using 
Calvert methods and materials. 



Evaluation Methods 

The multi-year, multi-method evaluation of the implementation of the Calvert 
curriculum at Woodson is similar in design to the evaluation conducted at Barclay. As at 
Barclay, implementation began with kindergarten and first grade, then added a grade every 
year as the initial cohort progressed. This gradual phasing-in has provided a low-cost, well- 
matched control group of the next older Woodson-pre-Calvert cohort of students in the same 
school. As at Barclay, this older cohort includes neighbors and often brothers and sisters of 
the students in the Woodson-Calvert cohorts. 

Data have included background information on all students (e.g., 100% African 
American, 90% free or reduced-price lunch), district and Abell grant budget information, and 
data on the school before the Woodson-Calvert Program began. Process data include 
extensive classroom observations in Woodson-Calvert classes (grade 1 during 1994-95, 
grades 1 and 2 during 1995-96, and grades 1, 2, and 3 during 1996-97), and in Woodson-not- 
yet-Calvert classes (grades 2 through 5 in 1994-95). Observations of training sessions and 
interviews with teachers, the principal of Woodson, the Head Master of Calvert, and 
representatives of The Abell Foundation have all been valuable components of the design. 
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The third-year Woodson-Calvert evaluation relied on detailed observations plus a 
series of outcome measures: student attendance, rates of referrals to special education 
programs, scores on the performance based Maryland School Performance Assessment 
Program (MSPAP), and scores from a norm-referenced test, the Comprehensive Testing 
Program III (Educational Testing Service, 1994), locally referred to as the ERB because it 
was developed for the Education Records Bureau. The ERB is a norm-referenced test 
developed for use in private schools. At Barclay the ERB provided data generally 
comparable to, but often slightly lower than, data provided by the Comprehensive Test of 
Basic Skills (CTBS/4) (CTB, 1990). During the first year of implementation at Woodson, 
the results of the CTBS/4 were used in the evaluation of the effectiveness of the Calvert 
Program. However, those data are no longer gathered because the state and the district 
discontinued administration of that test. 



This report documents the third year of the Calvert Program at Woodson Elementary. 
More detailed information about Woodson Elementary, the Calvert School, the Calvert 
Program, and the first two years of implementation at Woodson may be found in the first- 
and second-year evaluation reports (Stringfield & McHugh, 1995, 1996) and in McHugh and 
Spath (1997). 

Implementation at Woodson has been modeled on the Barclay experiment. As at 
Barclay, the implementation is proceeding incrementally. During the first year, the Woodson 
kindergarten and first grade were brought into the program; in year two, the second grade 
was added; in year three, the third grade was added. As of the third year, seven teachers were 
participating — one in kindergarten, two in first grade, two in second grade, and two in third 
grade. One grade per year is scheduled to be incorporated into the program until all six 
grades are using the Calvert curriculum during the 1999-2000 school year. 

Peg Licht, who had experience teaching in Baltimore City Public Schools and at 
Calvert and who had been the full-time coordinator of the Calvert Program at Barclay School 
from 1990 through June 1994, became the full-time Calvert facilitator at Woodson in 
September 1994. In this role she has overseen the day-to-day, on-site implementation of the 
program. She monitors, assists, coaches, models, and teaches. She ensures faithful replication 
of the Calvert curriculum and methods. She is simultaneously a coach, cheerleader, and 
watchdog. She has been an invaluable, fundamental, and vital component of the implementa- 
tion. 



Implementation 



ERIC 




A Guiding Committee is overseeing the implementation of the Calvert program at 
Woodson. The committee is chaired by Ms. Culman of the Abell Foundation and includes 
Mr. Hall and Sally Carr (Assistant Head) of Calvert, Dr. Spath, Ms. Licht, and Nancy 
Bethke, Woodson’s reading and math intervention teacher. The group meets every four to 
six weeks to discuss the progress and the problems of the reform. 

During the summer of 1996 the Woodson third-grade teachers and assistants attended 
a two-week training at Calvert School. The Woodson participants learned specifics about 
scheduling, weekly homework sheets, monthly report cards, parent conferences, and the 
particular written formats required in all student work. They were taught how to write in 
Calvert script and how to instruct in composition and spelling. Woodson teachers reviewed 
lessons in the Calvert correspondence program. Additionally, they learned the Calvert 
approach to literature, math and math manipulatives, athletics, poetry, science, spelling, and 
fine arts appreciation. In this summer training, developed and led by Ms. Licht, some of the 
presentations were made by the Calvert Head Master, Assistant Head, and teachers at Calvert 
and other schools. 

Third-year implementation of specific components of the Calvert Program — the 
corrections period, weekly homework sheets, composition instruction, folders, and 
instruction in spelling, science, reading, and mathematics — are discussed in the following 
sections. 

Corrections Period. Students in the Calvert Program use the 30 minutes prior to the 
first instructional period as a time to correct previous work, complete unfinished work, 
perfect folder papers, read independently, or do other instructionally related tasks. Common 
activities during this period are correcting spelling tests or dictations, finalizing composi- 
tions, catching up on incomplete work or work missed during an absence, or reading for 
pleasure. An adaptation that Calvert made at Woodson, as had been made at Barclay, was 
the daily assignment of specific Reading Work Pages (a supplement to the reading program) 
during the pre-school period. 

During the 1996-97 school year, five of the six Woodson teachers implemented the 
modified corrections period. The sixth teacher abandoned the Calvert design and instead 
assigned journal writing each morning. The only time she used the period as it had been 
designed was on Monday, when she had students correct errors they had made on the 
previous Friday’s spelling test. She continued these practices for most of the year until the 
Guiding Committee insisted that she conform to the design. 
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During the research group’s on-site observations, the most-often observed activity 
in grades one and two was the completion of Reading Work Pages. Roughly 40% of the first 
graders and 60% of the second graders were working on these reading worksheets. In third 
grade, the three activities which predominated were journal writing, spelling, and completion 
of unfinished work, each at about 20% of students observed. On-task rates during the 
corrections period averaged 70% across the three grades. Generally, an 80% on-task rate is 
associated with high achievement gains (Stallings, 1980). 

During the corrections period, the teacher and assistant are to be interacting with 
students or parents, not doing paperwork. During observations, teachers spent one-third of 
their time working one-on-one with students, 13% talking with parents, and 13% walking 
around the classroom, checking on the work of the children. Seventeen percent of their time 
was spent on paperwork. The assistants spent 65% of their time with students of which 45% 
was spent working with individual children, 10% leading the journal writing activity, and 
10% monitoring the work of the entire class. They talked with parents less often than 
teachers did — 3% of their time. Another 32% of their time was spent preparing supplies and 
papers for the day and monitoring students in the halls. 

In some classes, most often in first grade, parents and grandparents helped during the 
corrections period. They took down chairs, helped children with their coats and supplies, and 
listened to students read. They appeared to know many of the children, and they helped 
children other than members of their own family. In addition, the resource teacher assigned 
to the most academically needy students went to the homerooms of her students to check on 
their progress during the pre-school period. 

In general, Woodson exhibited solid implementation during the corrections period. 
It was an informal, yet productive, time. Teachers and assistants spent a large percentage of 
time working with and monitoring the progress of students. Parents and grandparents were 
welcome, and they made a contribution. If parents had questions or concerns, they were 
accommodated. The only area of concern was the off-task rate, which was somewhat higher 
than desired. 

Weekly Homework Sheet. On Monday morning of each week, the Calvert model 
calls for the homeroom teachers to distribute to each student a sheet listing the homework 
assignments for the entire week. This is part of school-parent communications. The parents 
are to see that the children do their homework and are to help their children study spelling 
words. 



The teachers in grades one through three did, in fact, prepare and send home on 
Monday the homework assignments for the entire week. The first- and second-grade teachers 
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prepared a more comprehensive package by including worksheets for math practice and, in 
the case of grade one, copies of the week’s spelling list and the dictation were sent home. 

Folders. Every month, Calvert students must compile specified work from the 
previous four weeks for inclusion in the monthly folder. The folder is considered a 
presentation piece and contains the weekly compositions, the weekly spelling tests, and 
representative samples of work in reading, mathematics, science, history, and geography. All 
work must be error free. The folders are sent home at the end of the month and are a part of 
school-parent communications. Before the folders go home, an adult reviews each child’s 
work with the child. 

The Woodson teachers have done a good job of coordinating the compilation of error- 
free papers. One problem that remains, however, is how to review each folder each month. 
Spending only five minutes with each child would require over two hours each month in 
each classroom. Woodson has experimented with several options, but a completely satisfying 
solution has proven elusive. 

Spelling Instruction. Students in the Calvert Program receive spelling instruction 
each week, although the time spent, types of lessons, materials used, and instructional period 
used vary by grade. In the early grades, spelling, composition, dictation, and writing practice 
are taught as a unit. There is much spillover between the subjects. For example, the spelling 
words introduced on Monday may be used as the basis for the weekly composition, the 
dictation, or the practice of writing in Calvert script. A spelling test is given to all students 
on Friday and is included in the monthly folder after corrections have been completed. In the 
upper grades, spelling may be incorporated more into reading, and a Calvert workbook, titled 
Vocabulary Connections, may be used in conjunction with the current reading material. 
Weekly spelling tests are also given to students in the upper grades. 

In each class in each grade, spelling was taught as specified in the Calvert Program. 
New spelling words were given on Monday and tested on Friday. Appropriate exercises were 
conducted throughout the week. Students corrected test errors on Monday, and the corrected 
spelling tests were added to the monthly folder papers. 

Composition Instruction. Beginning in January of first grade, all students in the 
Calvert Program write a composition each week. Typically, on Monday, the topic is 
introduced and discussed. On Tuesday, the class writes a composition together — children 
suggest sentences and the class decides which are the best. On Wednesday, the students must 
write a composition on their own. By Thursday morning, each teacher must have reviewed 
each child’s work, marking all mistakes, because the Thursday morning corrections period 
is used by the students to rewrite their compositions. By Friday morning, each teacher must 



again have reviewed all work. Further corrections to the compositions are made by the 
students during the Friday morning corrections period. Each composition is included in the 
monthly folder. All final compositions must be error-free. 

At Woodson, all students in the Calvert Program did, in fact, write a composition 
each week. In some classrooms the days on which specific steps in the writing process were 
conducted differed from the days specified by Calvert, but the deviation was not disruptive. 

Three concerns arose regarding composition instruction. In one classroom the 
compositions were almost identical. This is implausible if students were writing their own 
compositions. Second, one educator noted that third-grade students made errors that they 
should not be making (such as no capital at the beginning of a sentence) on MSPAP. Perhaps 
students were relying too much on teachers to correct punctuation and grammar mistakes 
during composition instruction, and an adaptation in the writing process should be identified 
and implemented. Third, the language of instruction and the types of compositions and topics 
do not correspond with MSPAP. Only minor modifications would be needed to fit the current 
composition instruction into MSPAP requirements and expectations. 

Science Instruction. In the third year of implementation, Woodson planned to adapt 
the Calvert science curriculum to incorporate the processes and outcomes which are tested 
on MSPAP. Teachers were required to write new science lessons with MSPAP in mind. They 
were to do so by using the Calvert science textbooks and some of the units of Science: 
Thinking, Application, and Research Skills (STARS; Baltimore City Schools, 1991). 

The initiative was planned and begun in the summer of 1996 by Ms. Bethke and 
Vanessa Merrick, a special education teacher. These two teachers wrote sample MSPAP-like 
questions for all grade levels from first through sixth and made vocabulary lists and outlined 
the MSPAP skills covered in the science text in grades one through three. They purchased 
and organized the required materials. An in-service in September led by Ms. Bethke 
introduced all Woodson teachers to the structure, rationale, and requirements of the STARS 
program. However, implementation was inconsistent. 

Reading and Mathematics Instruction. Students in the Calvert Program receive 
reading and mathematics instruction every day. The content of any one day’s lesson, of 
course, depends on the grade level and the group. (Calvert groups in reading and mathemat- 
ics according to performance.) In first through third grade, the reading books typically used 
are basals, Reading Work Pages, Reading Skill Builder booklets, extra reading books such 
as Smiling Hill Farm, and rhymes, verses, and stories. The Macmillan (1991) math series 
is used in all grades. 
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There are usually four reading and four math groups in each grade. At the allotted 
instructional time, the students go to their groups. Children are exchanged across classes, but 
not grades. The teachers and the assistants each teach one group, and instruction normally 
takes place in separate rooms. At Woodson, this model was followed. 



Outcomes 

To gauge if the Calvert Program was having a three- year effect on the public school 
children of Woodson Elementary, the evaluation team gathered data on various indicators. 
Analyses of data in the areas of student attendance, mobility, referrals to special education, 
and student achievement test scores are presented below. 

Student Attendance — Schoolwide 

In the past five years Woodson has aggressively worked to increase attendance by 
giving rewards to children who attend regularly and by contacting the parents of children 
who are frequently absent. During the 1996-97 school year, efforts were increased. Every 
four to six weeks there was a special event, such as a trip to the movies, bowling alley, or 
skating rink, for children with perfect attendance for a specified period of time. The school 
continued its previous practice of daily announcements of the classes which had perfect 
attendance the preceding day and identifying children or classes with perfect attendance in 
The Woodson Flyer, the principal’s weekly newsletter. The attendance monitor or the office 
secretary called the homes of every child absent each day and made home visits to parents 
who could not be contacted by telephone. These efforts have had a schoolwide impact. 
Woodson’s 1997 schoolwide attendance rate of 96.4% was among the highest in Baltimore 
City and merited the state-defined status of “excellent.” (In the Maryland School Perfor- 
mance Report, 1997, “excellent” is defined as “a highly challenging and exemplary level of 
achievement indicating outstanding accomplishment in meeting the needs of students.”) The 
“excellent” level had been obtained in 1994, and “satisfactory” status (at least 94% 
attendance) was reached in 1993, 1995, and 1996. 

Beyond meeting the state goal of “excellent,” Woodson’s ability to sustain high 
attendance rates is important because recent studies suggest that increased attendance has a 
positive influence on student performance (Caldas, 1993; Lamdin, 1996). The 1997 
Woodson attendance rate of 96.7% was well above the 1997 Baltimore City elementary 
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school average of 93.3%. These differences translate into six extra days of schooling each 
year for the Woodson students. 

Student Attendance — Woodson-Calvert Cohorts 

As can be seen in Table 1, there is very little difference between the attendance rates 
of the pre-Calvert cohort and the three Woodson-Calvert cohorts. In six of seven within- 
grade comparisons, those slight differences favor the Woodson-Calvert cohorts over the pre- 
Calvert cohort. 



Table 1 

Mean attendance rates for first, second, and third Woodson-Calvert cohorts and 
pre-Calvert cohort; first-, second-, and third-grade data. 


Cohort 


Grade 1 


Grade 2 


Grade 3 


Third Woodson-Calvert 


97.1% 


— 


— 


Second Woodson-Calvert 


95.4% 


96.8% 




First Woodson-Calvert 


94.2% 


95.3% 


97.0% 


Pre-Calvert Cohort 


95.2% 


94.5% 


95.4% 



Results from Norm-Referenced Achievement Tests 

As groundwork for the implementation of the Calvert Program at Woodson, the 
Calvert staff administered the Comprehensive Testing Program III test (locally referred to 
as the ERB) to Woodson students from the second through the fifth grade in the fall of 1994. 
In the springs of 1995, 1996, and 1997 Calvert tested students in grades one through five, 
again using the ERB. In the Barclay-Calvert experiment, longitudinal ERB results tended to 
parallel, but be more stable than, CTBS results (Stringfield, 1995). The ERB is a norm- 
referenced test used by private schools. 

The results of the reading comprehension, writing mechanics, and mathematics tests 
of the ERB are presented separately. All analyses were performed on normal curve 
equivalents (NCEs) 1 which were converted to percentiles after quantitative analyses had been 



Percentiles have the psychometric disadvantage of not forming a normal distribution and therefore being 
inappropriate for most psychometric analyses. Performing analyses using normal curve equivalents (NCEs) 
and then converting to percentiles for presentation avoids these psychometric problems. The NCE scale has 
a mean of 50 and a standard deviation of 21.06. NCE scores are identical to the percentile distribution at the 
values of 1 , 50, and 99. 





( 



completed. To be included in the data analysis for this report, the third-cohort students (first 
grade during the 1996-97 school year) must have been in first grade at Woodson continu- 
ously since September 30, 1996, and the first-cohort, second-cohort, and the pre-Calvert 
students must have been students at Woodson during each applicable test administration. 
For those students eligible to be included in the data analysis who were absent during one 
of the ERB subtests, the evaluators interpolated a score based on the average of their scores 
on the same subtest from other years. The grade-one data for the pre-Calvert cohort is from 
an ERB test administered to those students early in the fall of grade two (the earliest data 
available). 



Table 2 

Mean ERB Reading Comprehension national percentiles for Woodson’s pre-Calvert 
and Calvert cohorts, spring 1994-spring 1997. 


Grade 


Cohort 


(n) 


Mean 

Percentile 


x NCE 


SD 


Effect 

Size 


i 


Pre-Calvert 


(18) 


18 


30.4 


6.7 






First Woodson-Calvert 


(32) 


49 


49.5 


16.5 


+2.8 




Second Woodson-Calvert 


(29) 


40 


44.6 


18.0 


+2.1 




Third Woodson-Calvert 


(50) 


49 


49.6 


29.0 


+2.9 


2 


Pre-Calvert 


(18) 


22 


33.8 


10.6 






First Woodson-Calvert 


(32) 


48 


49.0 


20.6 


+ 1.4 




Second Woodson-Calvert 


(29) 


71 


61.7 


22.6 


+2.6 


3 


Pre-Calvert 


(18) 


25 


35.7 


8.2 






First Woodson-Calvert 


(32) 


46 


47.8 


18.7 


+ 1.5 



ERB Reading Comprehension Results. The reading comprehension subtest of the 
ERB tests students’ skills in vocabulary, recognizing explicit information, identifying 
explicit main ideas, analyzing, hypothesizing, and summarizing. As can be seen in Table 2, 
each Woodson-Calvert cohort, on average, has achieved substantially higher scores on the 
ERB Reading Comprehension test than the pre-Calvert cohort. In grade one, the Woodson- 
Calvert cohorts’ average national percentile scores of 49, 40, and 49 were from 22 to 3 1 
percentiles higher than the last pre-Calvert cohort’s mean score of 18th percentile. 

The right-hand column presents effect size computations. The computation is as 
follows: (Calvert mean NCE - Pre-Calvert mean NCE/Pre-Calvert Standard Deviation. An 
effect size larger than +.5 is usually considered large. The Calvert effect sizes for Reading 
Comprehension in grades 1-3 ranged from 1.4 to 2.9 and were quite large. The second-grade 
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data show an even wider range. The Woodson-Calvert students scored an average of 26 to 
49 percentiles higher than the pre-Calvert students mean score of 22nd percentile. The third- 
grade scores show a mean 21 percentile gap between the first Woodson-Calvert and the pre- 
Calvert students. 

ERB Reading Comprehension Results Expressed in Quartiles. The percentage 
of students scoring in each quartile in reading is shown in Tables 3, 4, and 5. The difference 
between the pre-Calvert cohort and the first two Woodson-Calvert cohorts is dramatic. In 
grade one, 72% of the pre-Calvert students scored in the lowest quartile, as compared to 
16%, 35%, and 6% of the Woodson-Calvert students. No pre-Calvert first-grade student 
scored above the second quartile, while 47%, 24%, and 42% of the first three Woodson- 
Calvert cohorts did. Grade two reading scores show similar disparities in the reading 
achievement levels of the students. While the pre-Calvert cohort made gains in the second 
grade, 44% still scored in the lowest quartile and only 6% scored in the third quartile. By 
contrast, 44% of the first Woodson-Calvert cohort and 76% of the second Woodson-Calvert 
scored in the upper two quartiles in reading in grade two. 



Table 3 

Percentage (and ns) of Woodson first-grade students, by cohort, scoring 
in each quartile of the ERB Reading Comprehension test. 


Cohort 


Quartile 


Lowest 


Second 


Third 


Highest 


Pre-Calvert 


72% (13) 


28% (5) 






First Woodson-Calvert 


16% (5) 


37% (12) 


28% (9) 


19% (6) 


Second Woodson-Calvert 


35% (10) 


41% (12) 


1 4% (4) 


10% (3) 


Third Woodson-Calvert 


6% (3) 


52% (26) 


24% (12) 


18% (9) 



Table 4 

Percentage (and ns) of Woodson second-grade students, by cohort, scoring 
in each quartile of the ERB Reading Comprehension test. 


Cohort 


Quartile 


Lowest 


Second 


Third 


Highest 


Pre-Calvert 


44% (8) 


50% (9) 


6% (1) 




First Woodson-Calvert 


22% (7) 


35% (11) 


25% (8) 


19% (6) 


Second Woodson-Calvert 


7% (2) 


17% (5) 


62% (18) 


14% (4) 
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Table 5 

Percentage (and /is) of Woodson third-grade students, by cohort, scoring 
in each quartile of the ERB Reading Comprehension test. 




Quartile 


Cohort 


Lowest 


Second 


Third 


Highest 


Pre-Calvert 

First Woodson-Calvert 


33% (6) 

25% (8) 


61% (11) 
37% (12) 


6% (1) 
16% (5) 


22% (7) 



Grade three results reveal gains for the pre-Calvert cohort and a decline for the first 
Woodson-Calvert cohort, compared to first- and second-grade results, but only 6% of pre- 
Calvert students scored in the third or highest quartile while 38% of the first Woodson- 
Calvert cohort did. 

ERB Writing Mechanics Results. The ERB writing mechanics subtest is 
administered for the first time in grade two. It is designed to measure students’ understanding 
of writing conventions, such as punctuation, capitalization, spelling, and appropriate use of 
language. Data in Table 6 show that in second grade the pre-Calvert cohort obtained an 
average writing mechanics score at the 36th percentile. By contrast, the first and second 
Woodson-Calvert cohorts scored at the 7 1 st and 67th percentiles, respectively. In grade three, 
the gap between the first cohort and the pre-Calvert cohort had decreased from 31 to 2 1 
percentiles, still a very large difference. All Woodson-Calvert means were above the national 
average, and the effect sizes were in the 1.9-2. 7 range. 



Table 6 

Mean ERB Writing Mechanics national percentiles for Woodson’ 
and Calvert cohorts, spring 1994-spring 1997. 


s pre-Calvert 


Grade 


Cohort 


(n) 


Mean 

Percentile 


x NCE 


SD 


Effect 

Size 


2 


Pre-Calvert 


( 18 ) 


36 


42.3 


7.1 






First Woodson-Calvert 


(32) 


71 


61.6 


15.1 


+2.7 




Second Woodson-Calvert 


(29) 


67 


59.3 


13.8 


+2.4 


3 


Pre-Calvert 


( 18 ) 


38 


43.9 


6.7 






First Woodson-Calvert 


(32) 


59 


54.6 


16.7 


+ 1.9 
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ERB Writing Mechanics Results Expressed in Quartiles. Scores on the ERB 
Writing Mechanics test show large differences between the pre-Calvert and Woodson- 
Calvert cohorts. As seen in Tables 7 and 8, only 1 1% and 17% of the pre-Calvert students 
scored in the third quartile in grades two and three, respectively. No pre-Calvert student 
scored in the highest quartile. In contrast, at 78%, 76%, and 44%, the Woodson-Calvert 
cohorts scored in the upper two quartiles over grades two and three. Of particular 
significance are the second-grade scores for the Woodson-Calvert cohorts — 78% of the first 
Woodson-Calvert cohort and 76% of the second scored at or above the 50th percentile. These 
are striking outcomes. 



Table 7 

Percentage (and ns) of Woodson second-grade students, by cohort, scoring 
in each quartile of the ERB Writing Mechanics test. 




Quartile 


Cohort 


Lowest 


Second 


Third 


* Highest 


Pre-Calvert 


22% (4) 


67% (12) 


11% (2) 




First Woodson-Calvert 




22% (7) 


44% (14) 


34% (11) 


Second Woodson-Calvert 


7% (2) 


17% (5) 


38% (11) 


38% (11) 



Table 8 

Percentage (and ns) of Woodson third-grade students, by cohort, scoring 
in each quartile of the ERB Writing Mechanics test. 


Cohort 


Quartile 


Lowest 


Second 


Third 


Highest 


Pre-Calvert 

Second Woodson-Calvert 


5% (1) 

3% (1) 


78% (14) 
53% (17) 


17% (3) 

19% (6) 


25% (8) 



ERB Mathematics Results. The emphasis in the ERB Mathematics subtest is on 
process and is based on the National Council of Teachers of Mathematics Standards for 
Curriculum and Evaluation (1989). The ERB mathematics scores are displayed in Table 8. 
The Woodson-Calvert students’ first-grade mathematics scores of 67, 62, and 61 are each at 
least 27 percentiles higher than the pre-Calvert students’ mean percentile score of 34. The 
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second-grade data are similar. The Woodson-Calvert students scored on average 28 
percentiles higher than the pre-Calvert cohort. In third grade the difference between the first 
Woodson-Calvert cohort and the pre-Calvert cohort decreased, but it remained at a dramatic 
21 percentiles. Note that, as with writing mechanics, all Woodson-Calvert ERB mathematics 
means have been above the national average, and that effect sizes ranged from 1.4 to 2.1, all 
very large. 



Table 9 

Mean ERB Mathematics national percentiles for Woodson’s pre-Calvert 
and Calvert cohorts, spring 1994-spring 1997. 


Grade 


Cohort 


(«) 


Mean 

Percentile 


x NCE 


SD 


Effect 

Size 


i 


Pre-Calvert 


(18) 


34 


41.3 


10.0 






First Woodson-Calvert 


(32) 


61 


55.8 


19.4 


1.4 




Second Woodson-Calvert 


(29) 


62 


56.4 


16.5 


1.5 




Third Woodson-Calvert 


(50) 


67 


59.4 


24.9 


1.8 


2 


Pre-Calvert 


(18) 


39 


44.1 . 


7.5 






First Woodson-Calvert 


(32) 


68 


59.7 


19.9 


2.1 




Second Woodson-Calvert 


(29) 


66 


58.6 


12.3 


1.9 


3 


Pre-Calvert 


(18) 


33 


40.8 


5.6 






First Woodson-Calvert 


(32) 


54 


51.9 


17.6 


2.0 



ERB Mathematics Results Expressed in Quartiles. The distinctions seen in the 
reading comprehension and writing mechanics tests can also be seen in the mathematics 
scores. Tables 10, 11, and 12 show the percentages of students scoring in each quartile for 
grades one, two, and three in mathematics. The Woodson-Calvert students scored 
significantly higher than the pre-Calvert students. No pre-Calvert student scored in the 
highest quartile in any year while 22% to 36% of Woodson-Calvert cohort students scored 
in the highest quartile for all grades and years. 
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Table 10 

Percentage (and /is) of Woodson first-grade students, by cohort, scoring 
in each quartile of the ERB Mathematics test. 




Quartile 


Cohort 


Lowest 


Second 


Third 


Highest 


Pre-Calvert 


28% 


(5) 


61% (11) 


11% (2) 






First Woodson-Calvert 


6% 


(2) 


38% (12) 


34% (11) 


22% 


(7) 


Second Woodson-Calvert 


3% 


(1) 


41% (12) 


28% (8) 


28% 


(8) 


Third Woodson-Calvert 


14% 


(7) 


38% (19) 


12% (6) 


36% 


(18) 



Table 11 

Percentage (and ns) of Woodson second-grade students, by cohort, scoring 
in each quartile of the ERB Mathematics test. 




Quartile 


Cohort 


Lowest 


Second 


Third 


Highest 


Pre-Calvert 


1 1% (4) 


67% (12) 


22% (2) 




First Woodson-Calvert 


3% (1) 


38% (7) 


28% (14) 


31% (11) 


Second Woodson-Calvert 




17% (5) 


59% (11) 


24% (11) 



Table 12 

Percentage (and ns) of Woodson third-grade students, by cohort, scoring 
in each quartile of the ERB Mathematics test. 


Cohort 


Quartile 


Lowest 


Second 


Third 


Highest 


Pre-Calvert 

First Woodson-Calvert 


22% (4) 

3% (1) 


72% (13) 
62% (20) 


6% (1) 
13% (4) 


22% (7) 



ERB Summary 

Three years of ERB testing have yielded strikingly similar results — Students 
involved in the Woodson-Calvert Program are achieving at much higher levels than their 
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older Woodson-pre-Calvert peers. Third year effect sizes for third grade were all quite large, 
ranging from 1.5 to 2.0. The effects are consistent with qualitative observational data 
gathered through the three years at Woodson. The size of the effects resemble second- 
through fourth-year findings from the Barclay-Calvert project (Stringfield, 1993, 1995). 
After three years the first Woodson-Calvert group to complete third grade, compared to pre- 
Calvert students, was achieving at a level that averaged 25 percentiles higher in reading, 21 
percentiles higher in mathematics, and 21 percentiles higher in writing mechanics. 

MSPAP Results 

MSPAP is a “next generation” performance-based testing program. The test is given 
to all third, fifth, and eighth graders across the state of Maryland. A total of approximately 
150,000 students take the test each year (Yen & Ferrara, 1997). MSPAP covers six content 
areas — reading, writing, language usage, mathematics, science, and social studies. The first 
four are defined by Yen and Ferrara (1997) as follows: 

Reading. The reading domain is defined by three purposes for reading: reading for 
literary experience, for information, and to perform a task. (p. 62) 

Writing. The writing domain is defined by three purposes for writing — to inform, 
persuade, and express personal ideas — and steps in the writing process — prewriting/plan- 
ning, drafting, revising, and proofreading, (p. 63) 

Language Usage. The single language usage outcome incorporates correctness and 
completeness features in the appropriate use of English conventions (e.g., punctuation, 
grammar, spelling) across a variety of writing purposes and styles, (p. 63) 

Mathematics. The mathematics domain is defined by nine content outcomes and 
four process outcomes. The Maryland outcomes are a close adaptation of the widely known 
NCTM Curriculum and Evaluation Standards for School Mathematics (National Council of 
Teachers of Mathematics, 1989).... The MSPAP open-ended mathematics tasks require 
students to solve multistep problems; make decisions and recommendations; communicate 
their ideas, understanding, and reasoning in mathematics; and explain processes they used 
to solve problems, (p. 64) 

The final two areas are: 

Science. The science domain covers the content areas of life science, physical 
science, and earth/space science and four process outcomes which include interpreting and 
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explaining information, demonstrating ways of thinking inherent in science, using the 
processes of science, and applying science to solve problems. 

Social Studies The social studies domain encompasses the content areas of political 
systems, geography, national and world history, and economics and the process outcomes 
of gathering, interpreting, and explaining information, demonstrating positive self-concept 
and empathy toward others, and expressing appropriate understanding and attitudes. 

The Maryland State Department of Education has resisted efforts by parents, 
researchers, and others to obtain MSPAP data at the individual student level. Thus, unlike 
in the ERB analyses, students new to the school could not be separated out. Thirty percent 
of the third-grade students who took the MSPAP at Woodson during the spring of 1997 
transferred into Woodson at some time in second or third grade. Because these students have 
not received three full years of the Calvert Program, MSPAP becomes a conservative test of 
the effects of the intervention. 



Table 13 

Percentage of third-grade students scoring at least at the satisfactory level 

on the six tests of MSPAP. 


Cohort (Year of Test) 


Reading 


Writing 


Lan- 

guage 

Usage 


Math 


Social 

Studies 


Science 


First Woodson-Calvert (1997) 


10.4 


20.8 


35.4 


8.5 


4.2 


10.4 


Pre-Calvert (1996) 


11.1 


13.3 


11.1 


0.0 


2.2 


0.0 


Baltimore City (1997) 


11.8 


14.6 


23.2 


10.8 


9.5 


10.7 


Maryland (1997) 


36.8 


40.0 


49.5 


41.4 


35.8 


38.2 



The results of the 1997 administration of MSPAP for third graders at Woodson, in 
Baltimore City, and in Maryland are shown in Table 13. Also included are the 1996 third- 
grade results for the pre-Calvert cohort. In all tests except reading, the first Woodson-Calvert 
cohort outscored the pre-Calvert students. Although the percentage of students eligible for 
free lunch is over 20% higher at Woodson than in the city generally, on the writing and 
language usage tests, the first Woodson-Calvert cohort outscored other Baltimore City 

'j 

students. In no tests did the Woodson children outscore the mean for all Maryland schools. - 



2 

Recall that the state mean free lunch rate is one-third that of Woodson. 



Even though the first cohort outscored their same-school, previous-program peers in 
all areas except reading, the results of the MSPAP analyses must be examined with caution. 
While language usage and writing scores indicated substantial advances against previous 
cohorts and the Baltimore City average, scores in the other areas did not reflect the 
remarkable gains seen on the ERB. The reading and mathematics results were surprising 
given the emphasis these subjects receive in the Calvert Program and the ERB scores 
reported earlier. 



Summary 

Woodson Elementary School has completed three full years of implementing the 
Calvert School curricular and instructional program. Third-year results have been very 
positive for an educational innovation. Level of implementation observed in classrooms, 
while allowing room for improvement, has been good. The quality of work in students’ 
folders has been high. Given that many of the parents at Woodson are financially unable to 
provide many of the educational opportunities and resources available to virtually all Calvert 
students, these are remarkable achievements. 

Student attendance is high throughout Woodson Elementary School, averaging 
96.4%. Referral rates to special education remain low. End of third-grade ERB scores in 
reading comprehension, mathematics, and writing mechanics were all dramatically above 
those of the previous cohort of Woodson students. The Woodson-Calvert students scored 
between 21 and 49 percentiles higher than the pre-Calvert students on the ERB. These 
differences are highly educationally significant, with effect sizes ranging from 1 .4 to 2.9. The 
MSPAP scores also indicated growth though rate of growth appeared to be less dramatic than 
was seen on the ERB. The Woodson-Calvert third-grade students appear to be, on average, 
much better prepared for success in the fourth grade and above than were their neighborhood 
peers in Woodson’s pre-Calvert cohort. 

The clearest conclusion that can be drawn from Woodson Elementary is that the 
Calvert curricular and instructional program, when implemented with determination and 
drive, can make a dramatic difference in the educational lives of young, urban children. 
Taken as a set, the program’s clear goals, intensive professional development model, 
rigorous reading program of phonics and interesting books, traditional rigorous mathematics 
program, daily focus on writing, and detailed monthly reports to parents, that include 
multiple writing, spelling, and mathematics products, can produce substantial improvements 
in student outcomes. These kindergarten through third-grade results leave little doubt that 
impoverished urban children, given appropriate curriculum and instruction, are capable of 
achieving at levels that are much higher than current urban averages. 
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